The field of electronic design automation (EDA) is well established. A number of software tools are used to describe a circuit at various levels of granularity or specificity. Such tools include gate level descriptions, which specify the circuit in very great detail, to high level descriptions written in hardware description languages such as Verilog or VHDL. The process of verifying a design through a simulation model of the device is aided by the availability of Verilog and VHDL. These languages are designed to describe hardware both at higher levels of abstraction and as gates or transistors, thus enabling designers to describe the elements and connections between elements of a circuit. Modern circuits have many millions of transistors, so it is essential to use some sort of design tools just to manage the complexity of the design, particularly for design verification.
Design verification is the process of determining whether an integrated circuit, board, or system-level architecture, exactly implements the requirements defined by the specification of the architecture for that device. Design verification for a device under testing (DUT) may be performed on the actual device, or on a simulation model of the device. For the purposes of explanation only and without intending to be limiting in any way, the following discussion centers upon testing which is performed on simulation models of the device.
As designs for different types of devices and device architectures become more complex, the likelihood of design errors increases. However, design verification also becomes more difficult and time consuming, as the simulation models of the design of the device also become more complex to prepare and to test.
The problem of design verification is compounded by the lack of widely generalizable tools which are useful for the verification and testing of a wide variety of devices and device architectures. Typical background art verification methods have often been restricted to a particular device having a specific design, such that the steps of preparing and implementing such verification methods for the simulation model must be performed for each new device.
As previously described, the process of verifying a design through a simulation model of the device is aided by the availability of hardware description languages such as Verilog and VHDL. The resultant simulated model of the device can receive input stimuli in the form of test vectors, which are a string of binary digits applied to the input of a circuit. The simulated model then produces results, which are checked against the expected results for the particular design of the device. However, these languages are typically not designed for actual verification. Therefore, the verification engineer must write additional programming code in order to interface with the models described by these hardware description languages in order to perform design verification of the device.
Examples of testing environments include static and dynamic testing environments. A static testing environment drives pre-computed test vectors into the simulation model of the DUT and/or examines the results after operation of the simulation model. In addition, if the static testing environment is used to examine the results which are output from the simulation model, then errors in the test are not detected until after the test is finished. As a result, the internal state of the device at the point of error may not be determinable, requiring the simulation to be operated again in order to determine such internal states. This procedure consumes simulation cycles, and can require the expenditure of considerable time, especially during long tests.
A more useful and efficient type of testing is a dynamic testing environment. For this type of environment, a set of programming instructions is written to generate the test vectors in concurrence with the simulation of the model of the DUT and while potentially being controlled by the state feedback of the simulated device. This procedure enables directed random generation to be performed and to be sensitive to effects uncovered during the test itself on the state of the simulation model of the device. Thus, dynamic test generation clearly has many advantages for design verification.
Within the area of testing environments, both static and dynamic testing environments can be implemented only with fixed-vector or pre-generation input. However, a more powerful and more sophisticated implementation uses test generation to produce the environment, particularly for functional verification in order for the various elements be defined and connected together correctly in order to have the circuit perform as specified. Specman Elite™, software developed by Verisity Ltd. in Israel and available through Verisity Design, Inc. in Mountain View, Calif., is the market leader in providing functional verification. Certain attributes of the software are described in copending, commonly assigned U.S. patent application Ser. No. 09/327,966, entitled “System and Method for Measuring Temporal Coverage Detection”, filed Jun. 8, 1999, and incorporated herein in full by reference. Useful background information is presented in copending, commonly assigned U.S. patent application Ser. No. 09/020,792, filed Feb. 6, 1998, entitled “Method and Apparatus for Test Generation During Circuit Design.”
The test generator disclosed in U.S. patent application Ser. No. 09/020,792 interacts with, and sits as a higher level over, such hardware description languages as Verilog and VHDL. The test generation procedure is written in a hardware-oriented verification specific object-oriented programming language. This language is used to write various tests, which are then used to automatically create a device verification test by a test generator module. A wide variety of design environments can be tested and verified with this language. Thus, the disclosed procedure is generalizable, yet is also simple to program and to debug by the engineer.
However, the reliance on human intervention is still highly problematic, particularly for increasing the coverage of the tests. “Coverage” is a function of the ability to fully explore the reachable state space of the design, in order to examine all possible functions and outcomes of any particular input or set of inputs to the DUT. Although the coverage of these tests can be increased manually, this is clearly a disadvantage, as it requires extensive input by the engineer. Therefore, a number of methods for attempting to automatically increase coverage are known in the art.
There are generally two classes of “coverage boosters” known in the background art: structural analysis or “white box” systems and knowledge base systems. Structural analysis based coverage enhancement derives coverage from the structure of the DUT, most typically from a gate level implementation of the DUT. A goal such as “toggle each node” is picked as the coverage goal and the test generator is most often based on various search algorithms operating on a representation of the DUT structure. These techniques were first introduced to identify “stuck at” faults in manufactured circuits. An example would be a production test generator such as HITEC from the University of Chicago (Chicago, USA; as described in T. Niermann and J. H. Patel, “HITEC: A Test Generation Package for Sequential Circuits,” European Design Automation Conf., Amsterdam, Netherlands, pp. 214-218, February 1991).
An example of a “white box” system is described with regard to U.S. Pat. No. 6,141,630, owned in common with the present application. This patent teaches a test bench for stimulating a simulated design with test vectors, after which the result is interpreted as though the design is a state diagram composed of blocks and transition arcs.
More recently there have been some such systems which operate on a higher level (for example Register Transfer Level). In essence such systems are based on the same techniques and most often synthesize the DUT into gates so that structural reasoning can be applied.
The shortcomings of all such methods are as follows. First, there is a lack of reference, since the coverage goals are taken from the DUT alone, without taking into account the specification requirements or the constraints imposed by the environment in which the DUT should function. This kind of testing fails to answer the most crucial question, which is whether the DUT actually functions as specified.
Second, with regard to capacity limitation, since the test generation algorithm requires an analysis of the DUT, the size of the DUT (or more accurately the state space projected by the state variables used to implement the DUT) must be limited. Current techniques are typically limited to small sub-sections of industrial designs.
Knowledge base systems are the second type of attempt to automatically increase coverage. In these cases the system has some knowledge of the specification of the DUT or the class of devices the DUT belongs to (CPUs for instance). Coverage goals are set according to the expected architectural and micro-architectural features. The knowledge base stores abstract tests aimed at various features. Such tests are applied to boost coverage where needed. The most prominent example of such system is Gensys from IBM (International Business Machines, USA), as disclosed in U.S. Pat. No. 5,202,889.
The major drawback of such a system is the need to construct such a knowledge base. The cost of constructing the knowledge base exceeds the manual work needed to test a specific DUT, so such an approach is viable only in cases where the knowledge base can be amortized over many generations of devices that are similar in function.